President’s Page

This article originally appeared in The Bar Examiner print edition, Spring 2023 (Vol. 92, No. 1), pp. 3–4.By Judith A. Gundersen

An NCBE Anniversary, NextGen Bar Exam Developments, and Thoughts on ChatGPT-4

The calendar says spring is here, but Madison seems firmly entrenched in winter. As I write this in late March, we just had a record-­breaking ­snowfall. But with daylight saving time and the spring equinox behind us, I know better weather is on its way.

We recently marked our 15th year in our Bedford Street headquarters in Madison. I remember moving day very well: February 14, 2008. The bar exam was just around the corner, and we had a brief window of time to ensure that all our systems were up and running and that we were ready to support exam administration, MBE scoring, and MEE/MPT grader training—a transition we managed well.

The brand new building we moved into in 2008 can no longer be described as such, but it has aged well. And although it has gone from seeing its walls filled to near capacity pre-pandemic to its current hybrid model occupancy, with team members spread out from coast to coast, it remains the vital headquarters for the work NCBE does in fulfilling its mission.

NextGen Bar Exam Developments

Just as we planned and built our NCBE headquarters with great care to create an updated physical space from which to serve our stakeholders, so too are we planning and building the next generation of the bar exam with great care and expertise to serve the profession. Exciting upcoming developments in the NextGen exam include publishing the Content Scope Outlines, outlining the projected exam length,  expanding pilot and field testing, and engaging in stakeholder outreach. In fact, I write this having just returned from a visit to South Dakota to meet with stakeholders there. My colleagues are also traveling or appearing virtually before Courts, Boards, and law schools. We are convening a Test Rubrics Advisory Committee in April to assist in creating and vetting grading materials for new item types.  

As we continue to hit these NextGen milestones, we are pleased and encouraged by advancements in test development, technological advances, stakeholder outreach, and new item performance among pilot testers.

ChatGPT-4 and the Bar Exam

Speaking of performance, recent news coverage about ChatGPT-4’s performance on the bar exam has garnered much attention. Though NCBE was not involved with the study that claims high achievement by ChatGPT-4 on the Uniform Bar Exam,1 the fact that the latest generation of artificial intelligence (AI) excelled on the exam is not that surprising, despite all the headlines. 

I think all of us in the legal profession are excited by the potential that AI brings to the profession, yet cautious about the implications. How might it help lawyers better serve clients? How might it be used to bridge the access-to-justice gap? What guardrails do we, as a society, need in place to ensure that content generated by AI is true and not harmful? Or how can we at least be sure that those who receive AI-generated legal advice know how to curate and discern what is accurate?

A recent article about ChatGPT notes that it “likes to fabricate things.”2 The experience of a colleague of mine seems to support this: they used GPT-4 to answer a legal research question, and although GPT-4 answered the question correctly, it then provided a nonexistent (or at least unfindable) case to support its correct answer.

I think the following quote sums up the huge potential upside to using this fast-changing and improving technology: “As the demand for better, faster, and more affordable legal services is only increasing in society, the need for supporting technology is becoming more acute. Further research on translating the capabilities of LLMs [large language models] like GPT-4 into real public and private applications will be critical for safe and efficient use.”3

Lawyers possess a great mix of qualities: empathy, judgment, and insight, among others. And we build relationships of trust and respect with clients who come to us regarding important life matters. I don’t see those human capabilities being replaced by AI anytime soon, but I do look forward to discovering the ways we can all use AI for the benefit of the profession and society.

2023 Annual Bar Admissions Conference

This year, the bar admissions community will be traveling to Nashville for our Annual Conference. We had originally booked Nashville for 2021, but that live event was canceled due to the ongoing pandemic restrictions, so we are trying again. And this time, we are competing for hotel space with Taylor Swift fans.

Our own “headliner” for this Conference is Bryan Stevenson, the highly acclaimed public interest lawyer and founder of the Equal Justice Initiative (EJI), a human rights organization in Montgomery, Alabama. Last May, our Board of Trustees and supporting staff members met in Montgomery, and we visited the EJI Museum and Memorial. It was the experience of a lifetime for us all. And Mr. Stevenson met with our group. That visit and meeting inspired us to invite him to be the keynote speaker at our Nashville conference. We are so grateful to him for agreeing and are excited for all attending to hear about his work and the work we all must do as lawyers and guardians of the Constitution to provide justice for all.

Until the next issue,

Judith A. Gundersen

Contact us to request a pdf file of the original article as it appeared in the print edition.